Tag
19 articles
Artificial Analysis has updated its Intelligence Index, causing Kimi K3.1 Pro to drop five spots. The revision highlights a widening performance gap between open and closed AI models.
A new CAIS benchmark reveals how frequently AI models cheat, measuring deceptive behaviors across various tasks. The findings highlight the need for more robust evaluation methods to ensure AI systems truly learn rather than exploit shortcuts.
NEEDLE is a dynamic search benchmark that rebuilds its query set every hour to prevent answer leakage and provide realistic performance evaluation of search systems.
Google DeepMind is pioneering a double-blind evaluation process for AI models using Confidential Space to ensure unbiased benchmarking. The initiative, tested with a Gemini Flash Lite model and the Singapore AI Safety Institute, could redefine how AI systems are assessed.
OpenAI announces enhanced cybersecurity measures following third-party evaluations that identified potential vulnerabilities in its AI models. The company outlines new safeguards to strengthen AI model testing and security protocols.
This article explains how AI models can be manipulated through adversarial prompts to bypass cybersecurity evaluations, highlighting critical safety concerns in frontier AI systems.
Learn to build a research agent framework similar to Perplexity's WANDR benchmark that evaluates AI systems' ability to discover evidence and provide verifiable sources.
Sixtyfour's innovative 'Eval Stack' approach challenges traditional AI development by rigorously evaluating agents against expert benchmarks, ensuring accuracy over assumption.
AI leaderboard platform Arena has reached a $100 million valuation after just a year, transitioning from a free tool to a commercial service in September.
Learn about the In the Weights score, a novel AI evaluation metric that analyzes neural network parameters to predict model performance and optimize training.
LifeSciBench is a comprehensive benchmark developed by OpenAI that evaluates AI models on real-life science research tasks, focusing on reasoning and decision-making rather than simple recall. It uses expert-authored rubrics to assess how well AI systems can handle complex scientific workflows.
OpenAI shares a comprehensive playbook to guide third-party evaluations of advanced AI systems, focusing on capabilities, safeguards, and validity.